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Abstract 



A display system receives (83) a data stream having data related to internet universal 
resource locator (URL) data in a data region of the frame region for continuous image frame data 
in the frame region, and displays continuous frames from the image frame data on a display 
monitor. The related data have one or several image entities in continuous frames correlated with 
one or several URLs; when the viewer selects the related image entity being displayed (99), the 
system accesses the internet (103); connection is made to the information source on the internet 
related to the URL, and the web page from the information source is downloaded (109) and the 
web page is displayed during the display process (111). Also, the viewer may access the related 
information while mutually conversing with the displayed web page. In order to display the 
relationship with a hidden URL, the entity may be highlighted during the display process. 



1. A type of varying signal that provides a display on a display monitor equipped with 

a first region containing image information with respect to a single frame for display, and 
a second region that contains information that correlates the visual entity being displayed 

provided by the information of said first region to an internet universal resource locator (URL) 

and that is separated from said first region. 

2. The varying signal described in Claim 1, characterized by the fact that said second 
region also contains information that correlates said visual entity to a prescribed position of said 
single frame display. 

3. The varying signal described in Claim 1, characterized by the fact that an analog 
television signal is contained in said first region. 



Claims 



[Names and addresses of the applicant and inventor are transliterations.] 
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4. The varying signal described in Claim 1, characterized by the fact that digital 
television data are contained in said first region. 

5. The varying signal described in Claim 1, characterized by the fact that it also contains 
information of said second region appropriate for visually highlighting said visual entity during 
said display. 

6. The varying signal described in Claim 5, characterized by the fact that highlighting of 
said visual entity is performed by luminance and/or contrast. 

7. The varying signal described in Claim 5, characterized by the fact that highlighting of 
said visual entity includes change in a color signal at the position of said visual entity. 

8. A type of system for TV broadcast transmission of television programs and display 
characterized by the following facts: 

the system has an information source for feeding a data stream, which has image frame 
data in one region and has the internet universal resource locator (URL) related to the image 
entity being displayed and provided by said image frame data in another region, 

and a receiving/displaying device that receives said image frame and displays it on a 
display monitor and also has an internet browser; 

said receiving/displaying device removes said URL from the region separated from the 
region containing said image frame data, starts said internet browser, accesses the worldwide 
web homepage related to said URL, and displays said homepage on said display monitor. 

9. The system described in Claim 8, characterized by the fact that said URL is related to 
the image entity in the frame displayed from said image frame data. 

10. The system described in Claim 8, characterized by the fact that said more than one 
entity being displayed is related to the URL. 

1 1. The system described in Claim 9, characterized by the fact that said image entity is 
highlighted during said display to show the viewer that said image entity is related to the URL. 

12. The system described in Claim 10, characterized by the fact that said highlighting is 
performed by means of adjustment of the luminance or contrast, by means of color adjustment, 
or by adding contour to said entity. 

13. The system described in Claim 8, characterized by the fact that said information 
source is a broadcast transmitter that broadcasts TV signals. 

14. The system described in Claim 8, characterized by the fact that said information 
source is a VCR that reproduces a video cassette recorder (VCR) tape. 

15. The system described in Claim 8, characterized by the fact that said information 
source is a CD-ROM drive that reproduces a CD-ROM disk. 

16. The system described in Claim 8, characterized by the fact that said information 
source is equipped with a computer device that retrieves the data stream from a hard disk drive. 
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17. The system described in Claim 8, characterized by the fact that said web page is 
displayed as an adjustable window that can be moved on said display monitor. 

18. A type of receiving/displaying device characterized by the following facts: 
it has a computerized controller, 

a data stream receiving/processing circuit, 

an internet browser, 

a display circuit, 

and a display monitor; 

said receiving/processing circuit receives a data stream that has image frame data and an 
internet universal resource locator (URL) that identifies the internet information source, and 
feeds said image frame data to said display circuit and said URL to said controller; 

said display circuit displays said image frame data on said display monitor; 

and said controller accesses said internet, makes said browser access said internet 
information source, and downloads and displays the web page from said internet information 
source. 

19. The device described in Claim 18, characterized by the fact that said URL is related 
to the prescribed entity image being displayed. 

20. The device described in Claim 18, characterized by the fact that it contains plural 
URLs related to different prescribed entity images being displayed. 

2 1 . The device described in Claim 1 8, characterized by the fact that said web page is 
displayed as an adjustable window that can be moved on said display monitor. 

22. A type of signal preparing/transmitting device characterized by the fact that it has 

an image device that monitors a scene and prepares a signal data stream containing image 
frame data from the scene, 

and a controller that adds an internet universal resource locator (URL) to said data stream 
and correlates said URL to the image entity in one or several said image frames. 

23. The device described in Claim 22, characterized by the fact that said controller adds 
more than one URL related to different image entities. 

24. The device described in Claim 22, characterized by the fact that said controller adds 
data to said data stream and visually highlights said image entity being displayed provided by 
said data stream. 

25. The device described in Claim 24, characterized by the fact that said highlighting is 
performed by means of adjustment of one or more of luminance, contrast and color. 

26. A method that correlates an entity in a broadcast image to an internet universal 
resource locator (URL), characterized by the following steps: 
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(a) a step in which a data stream having frame data regions containing sequential frame 
data is prepared from a scene using a visual image device; 

(b) a step in which said data for URL are inserted in a second region between said frame 
data regions; and 

(c) a step in which in said second region between said frame data regions, the data that 
correlate a prescribed entity in an adjacent frame data region to said URL are inserted. 

27. The method described in Claim 26, characterized by the fact that steps (b) and (c) are 
executed while prerecorded image data are edited. 

28. The method described in Claim 26, characterized by the fact that steps (b) and (c) are 
executed automatically in a live broadcast by means of a computerized device adjusted by said 
visual image device. 

29. The method described in Claim 26, characterized by the fact that in order to highlight 
said entity display, there is also a step in which data are inserted in said second region. 

30. A method for correlating an entity in an image displayed from a broadcast data stream 
to an internet universal resource locator (URL), characterized by the following steps: 

(a) a step in which an image frame obtained from a data stream having image frame data 
is displayed, 

(b) a step in which data related to said URL are acquired from a data region separated 
from said image frame data and interleaved with said image frame data, and 

(c) a step in which said entity in continuous image frames is formed as said related data 
and is correlated with said acquired URL. 

31. The method described in Claim 30, characterized by the fact that it also has a step in 
which said entity in said continuous image frames is visually highlighted. 

32. The method described in Claim 31, characterized by the fact that said highlighting is 
performed by adjustment of color, luminance and/or contrast in said display. 

33. A method for accessing additional information related to an image entity in a video 
display, characterized by the following steps: 

(a) a step in which a dynamic universal resource locator (URL) transmitted between 
frames in a data stream containing an image frame for said video display is correlated with said 
image entity, 

(b) a step in which said entity on said video display is selected by means of user input, 

(c) a step in which the internet is accessed as a response to said user input through a 
network interface module (NIM), and connection is performed to the data information source 
related to said URL, 

(d) a step in which the web page is downloaded from said data information source, and 

(e) a step in which said web page is displayed on said video display. 



34. The method described in Claim 33, characterized by the fact that said web page is 
displayed as an adjustable window that can be moved on said video display. 

35. The method described in Claim 33, characterized by the fact that said user selects by 
using a pointer device in manipulating and selecting a cursor. 

36. The method described in Claim 33, characterized by the fact that plural image entities 
are correlated with unique URLs, respectively, and more than one web page is accessed and 
displayed by selecting more than one image entity. 

Detailed explanation of the invention 
Field of the invention 

The present invention pertains to a multimedia device and method. More specifically, the 
present invention pertains to a system that integrates internet access to TV transmission and 
display and provides an entity and similar data for display to a TV viewer. 

Background of the present invention 

Development of novel methods for distributing television and video display to end users 
and parallel development of the internet, the related worldwide web (WWW) and other 
computerized information systems have been pursued. As a result, efforts have been made to 
provide an information entertainment system with various systems unified and strengthened. For 
example, developers have introduced a unified system that combines TV and computer 
subsystems, so that a TV can be used as web browser. 

Among the computer system structural elements of CPU, memory, and the like, some 
elements are assembled in a TV set chassis. For such a system, conventional TV structural 
elements and circuits are assembled together with computer structural elements, and the ability to 
switch modes is provided to the user. Consequently, by means of an appropriately equipped 
system, the user can select an analog TV program, digital TV program, conventional cable TV, 
satellite TV, and pay TV from various information sources. Similarly, the user can browse the 
WWW and display a web page, converse with a system related to the on-screen field, and jump 
to related information, a database, or another web page. 

For several other systems, computer structural elements are set in an accommodating 
container known as a set-top box separate from the TV receiver. For a set-top box system, the 
provider is connected to a conventional TV set, so the end user has no need to purchase a new 
TV with built-in computer structural elements. This is an advantage for the provider. 

For said unified system, either one in a single accommodating container or said set-top 
box system, the user can perform input by means of a hand-held device similar to a conventional 
remote controller by making IR communication between the remote controller and the 
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conventional set-top box or the receiver in a unified TV set. For web browsing or another 
computer mode, a cursor is displayed on the TV screen, and the cursor can be manipulated by 
means of buttons on the remote controller. Selection buttons are also set on the remote controller, 
and the computer user can execute conventional functions just like with the buttons on a 
conventional mouse or trackball or another pointer device. 

The aforementioned set-top box and computer unified TV set usually have the input of a 
TV antenna (analog), cable TV (analog or digital), and the recently adopted direct satellite TV 
(digital), and a hard disk drive, CD-ROM drive or another large-capacity storage device is also 
connected. Video data are uploaded from said drive, and a function in displaying a dynamic 
result on the TV screen is exhibited. 

The present inventors observed the following fact: by means of a combination of 
computer technology (digital) and TV technology, many functions that are familiar to computer 
users can also be performed by TV users, too. For example, the ability to provide a text subtitle 
for TV display is significantly strengthened. Picture-on-picture computer technology is now 
available. By means of the picture-on-picture scheme, the independent TV display becomes an 
independent window or overlay window on the display screen. The independent window can also 
support display of analog TV programs on one window, computer games on another window, 
and a teleconference on a third window, that is, it can support display of different data sources. 

However, even when all of the available digital technologies are adopted, substantial 
separation is still present between the world of computers and the world of dynamic TVs. For 
example, consider the case when the end user demands certain information. For example, the 
viewer of a TV program may recognize an actor/actress. Although the viewer may not know the 
name of the actor/actress, the viewer may try to remember the name, or may want to collect 
further information about the actor/actress. 

In the prior art, in this case, the user must wait until the list of actors/actresses appears at 
the end of the program. Then, the user writes down the name of the actor/actress and switches 
the mode to the browsing mode (assume that the system is a unified system). Then, the system 
accesses the WWW and searches the name of the actor/actress, or the name of the studio, the 
name of the program or another reference to get the desired information. When the desired 
information can be used at a certain site on the web page or in a related database, the user can 
expend some effort in getting the desired information. 

There is another simple embodiment. Consider a TV advertisement. The viewer may be 
interested in an automobile or another specific item due to the advertisement, and may want to 
purchase the item or similar one. However, due to the fee charged for the network television 
advertisement and the resulting restriction on the time left for purchase after the advertisement 
time, detailed information that can be provided for purchasing the advertised item is definitely 
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restricted. Usually only the most prominent features can be provided, yet the purpose of the 
advertiser is to guide customers to their products, and they provide general price information. 

For example, assume that there is an advertisement of a model of automobile. There is no 
way to provide all data about options, models, features, prices, payment schemes, as well as local 
information for test driving the car. In addition, even if all details are provided, the viewer has no 
way to write all of them down. The viewer may try to find good printed matter issued as an 
automobile lease advertisement. 

In many cases, except for the case of TV shopping channels that are perceived to be 
unreliable by many people, a viewer who wants to buy an item that has been advertised on TV 
must expend significant effort to actually purchase the advertised item. In addition, many major 
manufacturers do not sell their products through TV shopping technology of the conventional 
viewer telephone participation type. 

It is clear that there is a demand for development of a method and a device that can unify 
web browsing technology and TV broadcast technology to ensure that a viewer can quickly 
accumulate a large quantity of information about an entity that interests the viewer due to 
viewing of an advertisement in a TV broadcast program. In addition, there is a demand for 
development of a method that will allow a viewer to quickly and easily access detailed 
information about products advertised to the general public, as well as about interesting persons 
or items from the TV broadcast program. 

Abstract of the invention 

As a preferred embodiment of the present invention, the present invention provides a type 
of varying signal that provides a display on a display monitor equipped with a first region 
containing image information with respect to a single frame for display, and a second region that 
contains information that correlates the visual entity being displayed provided by the information 
of said first region to an internet universal resource locator (URL) and that is separated from said 
first region. Said second region also contains information that correlates said visual entity to a 
prescribed position of said single frame display. The signal of the first region may be either an 
analog or digital signal. The second region also contains information of contrast, luminance or a 
color signal, or any combination of them. By changing said combination, it is possible to 
strengthen the visual effect while the visual entity is displayed. 

As another embodiment, the present invention provides a type of system for TV 
broadcast transmission and display of television programs, characterized by the following facts: 
the system has an information source for feeding a data stream, which has image frame data in 
one region and has the internet universal resource locator (URL) related to the image entity being 
displayed and provided by said image frame data in another region, and a receiving/displaying 
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device that receives said image frame and displays it on the display monitor and also has an 
internet browser; said receiving/displaying device removes said URL from the region separated 
from the region containing said image frame data, starts said internet browser, accesses the 
worldwide web homepage related to said URL, and displays said homepage on said display 
monitor. 

For said system, said URL is related to the image entity in the frame displayed from said 
image frame data. Also, for said system, said more than one entity being displayed is related to 
the URL, or said image entity is highlighted during said display to show the viewer that said 
image entity is related to the URL. Said highlighting may be performed by means of adjustment 
of the luminance or contrast, by means of color adjustment, or by adding contour to said entity. 

Said information source may be a broadcast transmitter that broadcasts a TV signal. Said 
information source may be a VCR that reproduces a videocassette recorder (VCR) tape. Also, 
said information source may be a CD-ROM drive that reproduces a CD-ROM disk. In addition, 
said information source may be equipped with a computer device that retrieves the data stream 
from a hard disk drive, and said web page may be displayed as an adjustable window that can be 
moved on said display monitor. 

As another embodiment, the present invention provides a type of receiving/displaying 
device characterized by the following facts: it has a computerized controller, a data stream 
receiving/processing circuit, an internet browser, a display circuit, and a display monitor; said 
receiving/processing circuit receives a data stream that has image frame data and an internet 
universal resource locator (URL) that identifies the internet information source, and feeds said 
image frame data to said display circuit and said URL to said controller; said display circuit 
displays said image frame data on said display monitor; and said controller accesses said 
internet, makes said browser access said internet information source, and downloads and displays 
the web page from said internet information source. 

For said receiving/displaying device, said URL is related to the prescribed entity image 
being displayed. The device contains plural URLs related to different prescribed entity images 
being displayed. For said device, as a preferable scheme, said web page is displayed as an 
adjustable window that can be moved on said display monitor. However, a display may also 
replace a preceding display. 

As another scheme, the present invention provides a type of signal preparing/transmitting 
device characterized by the fact that it has an image device that monitors a scene and prepares a 
signal data stream containing image frame data from the scene, and a controller that adds an 
internet universal resource locator (URL) to said data stream and correlates said URL to the 
image entity in one or several said image frames. Said controller may add more than one URL 
related to different image entities. For said device, said controller may add data to said data 
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stream and visually highlight said image entity being displayed provided by said data stream. 
Said highlighting is performed by means of adjustment of one or more of luminance, contrast 
and color. 

As another scheme, the present invention provides a method that correlates an entity in a 
broadcast image to an internet universal resource locator (URL), characterized by the following 
steps: (a) a step in which a data stream having frame data regions containing sequential frame 
data is prepared from a scene using a visual image device; (b) a step in which said data for URL 
are inserted in a second region between said frame data regions; and (c) a step in which the data 
that correlate a prescribed entity in an adjacent frame data region to said URL are inserted in said 
second region between said frame data regions. 

In this method, steps (b) and (c) may be executed while prerecorded image data are 
edited, or these steps may be executed automatically in a live broadcast by means of a 
computerized device adjusted by said visual image device. Also, in order to highlight said entity 
display, there may also be a step in which data are inserted in said second region. 

As another scheme, the present invention is a method for correlating an entity in an image 
displayed from a broadcast data stream to an internet universal resource locator (URL), 
characterized by the following steps: (a) a step in which an image frame obtained from a data 
stream having image frame data is displayed, (b) a step in which data related to said URL are 
acquired from a data region separate from said image frame data and interleaved with said image 
frame data, and (c) a step in which said entity in continuous image frames is formed as said 
related data and is correlated with said acquired URL. 

This method may also have a step in which said entity in said continuous image frames is 
visually highlighted, and said highlighting may be performed by adjustment of color, luminance 
and/or contrast in said display. 

As another scheme, the present invention provides a method for accessing additional 
information related to an image entity in a video display, characterized by the following steps: 

(a) a step in which a dynamic universal resource locator (URL) transmitted between frames in a 
data stream containing an image frame for said video display is correlated with said image entity, 

(b) a step in which said entity on said video display is selected by means of user input, (c) a step 
in which the internet is accessed as a response to said user input through a network interface 
module (NIM), and connection is performed to the data information source related to said URL, 
(d) a step in which the web page is downloaded from said data information source, and (e) a step 
in which said web page is displayed on said video display. 

In said method, said web page may be displayed as an adjustable window that can be 
moved on said video display, and said user may select by using a pointer device in manipulating 
and selecting a cursor. In said method, plural image entities may be correlated with unique 
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URLs, respectively, and more than one web page is accessed and displayed by selecting more 
than one image entity. 

The present invention, due to the various viewpoints and embodiments, can provide a 
high level of capability in a friendly manner, and it provides advertisers and promoters a novel 
tool for accessing information that has not formerly been provided by television or other video 
displays. Also, it can avoid a tedious task that takes a lot of time for the viewer when additional 
information is demanded for said display and representation. 

Brief description of the figures 

Figure 1 is a block diagram illustrating a set-top box used as the platform for embodiment 
of the present invention. 

Figure 2A shows a TV display as an example of embodiment of the present invention. 

Figure 2B is a diagram illustrating display of a data stream for transmitting the TV 
display in the embodiment of the present invention shown in Figure 2A. 

Figure 2C is a diagram illustrating an example indicating the web page in the window of 
the TV display showing the results of embodiment of the present invention. 

Figure 3 A is a flow chart illustrating the steps in preparing a TV broadcast program 
according to the embodiment of the present invention. 

Figure 3B is a flow chart illustrating the steps in which, according to the embodiment of 
the present invention, a TV broadcast is received with a dynamic URL, and access to the web 
page is started. 

Explanation of preferred embodiments 

According to the embodiment of the present invention, for the individual images of 
persons, objects, or the like displayed on TV, the viewer selects an image. As a result, by means 
of a method that calls out the linked URL, the universal resource locator (URL) is linked, and, by 
calling out the URL, it is guided to the web site where the information related to the image is 
provided. In many embodiments of the present invention, the image related to the URL is 
highlighted on the display by means of a special color or highlighted luminance (halo) or 
contour. Highlighting may also be performed in other forms. 

Figure 1 is a block diagram illustrating set-top box (11) used for embodiment of the 
present invention. Said set-top box (1 1) is more complicated than is required for embodiment of 
the present invention in several embodiments. It has more functions, and it plays the role as a 
good type of platform in use. 

Said set-top box (1 1) is used to receive a signal from any of various information sources, 
such as decoder/tuner (13), satellite link (15), cable TV line (17), and VCR input (16). In this 
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embodiment, 80486 CPU (19) provides the management and computing ability, and it is 
connected via link (21) to decoder/tuner (13). As another embodiment, another type of CPU may 
be used. Said decoder/tuner (13) receives and decodes a satellite signal on satellite link (15), TV 
signal on line (17), and VCR video signal on line (16). 

In addition to the CPU and decoder/tuner, set- top box (11) includes motion picture expert 
group (MPEG) decoder (25). It translates the data fed to set-top box (1 1) in MPEG I or MPEG II 
format or a future format, and feeds the signal from the data to drive the TV or video monitor. 
Here, MPEG refers to an industry standard protocol for video data, and the technology is well 
known. 

Said MPEG decoder (25) is connected by link (27) to CPU (19), and it is connected to 
decoder/tuner (13) by link (29). MPEG decoder (25) has a video graphics adaptor (VGA) chip 
containing video RAM (VRAM), and has output line (3 1) to chip set (33). The VGA chip set is 
appropriate for driving a TV screen or computer display screen via a conventional circuit. One 
may also use other types of video circuits. 

Analog telephone modem (35) is connected to telephone line (37) and CPU (19), and it 
provides communication to a remote site for a pay-TV transaction, diagnosis, or download 
service. ISDN interface (39), optionally in this embodiment, connects optical digital data cable 
(41) and CPU (19) for digital phone communication. This may also be used in place of analog 
modem (35) or in addition to analog modem (35). There is also an optional interface for compact 
disk read-only memory (CD-ROM) drive (43) and hard disk drive (45). The modem shown here 
in the figure is convenient for use, and it is an example of a network interface module (NIM) 
used for internet access. 

Said set-top box (1 1) has ROM (47) for a BIOS function or the like, and it also has 
dynamic random access memory (DRAM) (49). Operating code (48) stored in DRAM (49) 
provides the function for embodiment of the present invention, and it is received by any 
conventional method that can be executed while it can be accessed by CPU (19). Said operating 
code (48) contains a web browser and a routine for a prescribed functionality according to the 
embodiment of the present invention to be explained later. Said WWW browser is appropriate 
for accessing a server on the WWW, such as server (54) connected to link (37). Server (54) 
represents any web server dialed up through one of the modems related to the system shown in 
Figure 1. 

For specialists in the computer field, the communication link shown in Figure 1 with 
respect to set-top box (1 1) is an example of data flow. It is clear that it does not show the type of 
prescribed link or bandwidth. For example, the type of CPU being explained is well known for 
communication on a parallel bus and has a bus cycle controlled by a bus controller for industry 



13 



standards, such as an ISA bus, EISA bus or the like. One may also use a universal serial bus 
(USB) or another type of communication link. 

For the set-top box shown in Figure 1, said VGA circuit (33) has output links (20), (22) 
for driving TV (51) and/or computer display monitor (53), and one or both of them are displayed 
in various embodiments. There is also IR communication remote controller (63) that is 
appropriate for control and selection of a cursor by direction button (67) and selection button 
(69). IR communication remote controller (63) communicates with receiver (65) in the set-top 
box. 

As well known technology, data stream transmission for display of an image on a display 
screen is performed by transmission of identifiable frames. This is a conventional feature for the 
transmission of both a prestored data stream and for live transmission by a transmission source, 
and for both digital and analog transmission. In any case, the specific technology is well known, 
and a detailed description is not necessary in the present specification. 

In the transmission of each frame, as well known technology, the information is 
transmitted in the space between frames. For the digital transmission format, such data are 
expected, and since there is a blank line between frames in analog TV transmission, it may also 
be used for data transmission within space and time limits. This method is adopted for coding the 
information for creation of a color display of a black and white television display, such as a 
special caption for those with impaired hearing [sic]. 

In various embodiments of the present invention, one or several entity images in the 
transmitted frame are identified for a position and range in the frame, and the relationship with 
the WWW URL is determined. As a simplified example, for an advertisement of a brand of 
automobile, an icon or emblem may be displayed for each frame at a prescribed position in the 
frame. For example, said emblem may be a prescribed emblem adopted for an automobile brand, 
such as the well known Ford, Chevrolet, BMW or other emblems. The emblem is provided in the 
advertisement by any known conventional technology. Figure 2 A depicts simplified frame (55) 
having BMW emblem (57) in the frame at a prescribed position with a prescribed area range. 

In this embodiment, the BMW advertisement is a prerecorded advertisement indicating 
how many times it should be transmitted and displayed. It is a generally displayed advertisement 
between portions of a TV program. In the recording of the advertisement, for editing of the 
procedure for an existing prerecorded advertisement, the data are recorded such that they are 
transmitted between frames, the position and range of the BMW emblem in adjacent frames are 
identified, and the emblem is correlated with a prescribed WWW URL of the general form 
http://www.bmw.com. The URL is a locator on the WWW with respect to the home page 
provided on the web server maintained by BMW, and it is a site for accommodating additional 
detailed information with respect to what is provided in the transmitted advertisement. 
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As another embodiment of the present invention, different entities in the frame are 
correlated with different URLs, and the user is provided with different homepages on the web or 
different data positions on the same homepage. Just like an emblem, the automobile itself may be 
highlighted and correlated with an URL, and the URL of the automobile is guided to information 
of an up-to-date model. An example of this scheme is http:www.bmw.com/latestmodel. 

Figure 2B is a diagram illustrating an example of data stream (59). The data for first 
frame (61) and second succeeding frame (65) are separated by data region (63). Said data region 
(63) contains data for identifying the position and range of BMW emblem (57) in first frame 
(61). Next data region (67) succeeds frame (65), and, similarly, the position and range of the 
BMW emblem in the frame are identified. Similarly, the URL related to the emblem is 
transmitted. In this way, each frame has its URL related to the correlated data region having the 
data of position and range of the image. 

In a conventional receiving system that does not provide computer capability and does 
not unify with TV capability, since data are present between frames, the additional data 
containing the URL are not used, as they pass through in a transparent way. In such a 
conventional TV system, only the frame data are used. However, in the system of the type 
explained with reference to Figure 1 that has a computer function unified with the TV function, it 
is possible to use the data between frames in a special scheme. 

In a system that has a CPU and memory and has a special control routine according to the 
present embodiment, the BMW emblem is handled as a conversation screen region in a method 
well known to a user of an access web page. For the computer system that has the TV circuit 
unified, cursor (70) is displayed on the screen, and this cursor (70) is positioned by the user by 
means of position button (67) of hand-held remote controller (63) (see Figure 1). In another 
system, the cursor is controlled by means of a pointer device or another convenient scheme. 

Back to the example of the BMW advertisement, when the TV frame containing BMW 
emblem (57) is displayed, the viewer may start the cursor (one may also adopt a scheme in which 
the cursor is started automatically when the viewer manipulates one or another of position 
buttons (67)). According to the embodiment of the present invention, a special control routine is 
adopted, and data between frames are removed, and the control routine for identifying the 
position and area range of emblem (57) in the adjacent frames and the related URL is used. 

When the viewer is interested in getting additional information, the viewer may 
manipulate the cursor to touch the region of emblem (57) and then may press down one of 
selection buttons (69) on the remote controller to start a selection signal. When the cursor 
touches the BMW emblem and the selection signal is received, the system executes a browser 
routine to access the WWW, and the web server explained above and maintained by BMW on 
the WWW (see server (54) and modem (35) or (39) shown in Figure 1) is dialed up. The URL in 
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the data region between frames for TV transmission and related to the BMW emblem is the 
WWW address for said dial-up operation. 

Once the viewer starts the system of the present invention, connection is made to the web 
server of BMW, and action is performed with one of several methods. As an embodiment in this 
case, the TV display is stopped. Instead, the initial web page downloaded from the BMW server 
is displayed. As a preferable scheme, the TV display is continued, while as shown in Figure 2C, 
the downloaded web page is displayed on window (71) with respect to the TV display. In this 
way, window (71) is enlarged or contracted, and it can be moved on the screen, and can be 
closed anytime when required by the viewer. The conversation region on the window related to 
the additional information or the correlated web server, just like a page displayed on a computer 
monitor via a conventional web browser for a web page on a superimposed window, can be 
activated by means of cursor (70) and selection buttons (69). Scrolling capability is included by 
means of a scroll bar or the like as would be provided in a conventional scheme with a web page 
on the WWW. In several embodiments, a restriction on area is applied on the display rather than 
matching of the web page size to that of the display window, so that the web page can be 
maintained at a minimum size, and, when all web pages are displayed and the window is too 
small, the capability of scanning and panning as well known technology is provided for 
accessing an entire page. 

The BMW web page shown on window (71) in Figure 2C is the inlet of information for 
accessing a large quantity of information through the WWW for the TV viewer, and it cannot be 
used in the original TV advertisement. Examples of the contents of the information include 
colors that can be used (together with the example), style of the car body (together with the 
paint), performance data, detailed price list, available sales lease period, sites near the viewer for 
meeting a dealer for a test drive, and various other data. If the system has an information display 
function for a demonstration video, reproduction is also possible for persons who have 
downloaded the demonstration video for watching. An even more important feature is that the 
viewer can access an included order sheet. As a result, the process for purchasing the car from 
the dealer becomes a user/buyer friendly process without any difficulty. 

Assuming that the viewer can input text with a keyboard so that he can use the system 
freely in addition to simply accessing additional information, the bidirectional capability of the 
internet link allows transmission of information to the server by the viewer through an input field 
by means of web page access. 

The aforementioned example of presentation of a BMW automobile advertisement is a 
specific example for adoption in the range of the present invention. As a modified example of 
this embodiment, the image region related to the URL may be an image of an automobile rather 
than the BMW emblem. The active image region can be driven to move for display of the TV 
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advertisement, and, by tracking the position and range of the screen for the interframe data, it is 
possible to adjust the movement of the cursor in real time on a dynamic picture. 

In a method that also shows the viewer a related region for accessing information 
corresponding to the dynamic image in addition to tracking and adjusting the dynamic image, the 
dynamic image may be highlighted during display. This is realized by depicting a contour on the 
screen using a certain scheme, adjusting the luminance and/or contrast, variation in color, or their 
combination in any form. 

In this way, according to embodiment of the present invention, any image of the TV 
broadcast program can be correlated with a dynamic URL for accessing additional information 
pertaining to the image. 

As a minimum-scale embodiment, as explained above, the URL is correlated in the 
interframe region, and, when an image is selected, the URL is displayed as text information on 
the screen for notification of the viewer or copying. Also, the URL may be downloaded to a 
memory device for later print-out or browsing. 

The list of applications and embodiments of the present invention is almost unlimited. 
The aforementioned BMW advertisement is a commercial application, and one can imagine 
numerous such commercial applications for advertisement. 

As another embodiment, rather than an advertisement, an actor/actress may be correlated 
with an URL in a TV program. This URL guides the viewer to a web page that contains 
information indicating roles of the same actor/actress in other movies, biographical information, 
as well as applicability and selection of additional products (agents or the like). As another 
example, for politics, the present invention allows access from a political spot having an active 
region from the standpoint of the focus of an argument, the site of an upcoming rally or other 
activities. On the other hand, for sporting events, information for a prescribed athlete may be 
provided, and schedules, upcoming games, records, play-off position possibilities and similar 
information can be provided. Also, sports associations can provide season tickets prices and 
similar information, and tickets may be sold via internet links. 

Figure 3 A is a flow chart illustrating a procedure of operation with a device together with 
a data stream containing a dynamic URL correlated with an embodiment of the present 
invention. 

In step 83, reception is performed for a data stream that transmits entity data and one or 
several dynamic URLs in a data region separated from the image frame data. Said one or several 
URLs are linked to the image entity by means of tabs. An URL is transmitted before (ahead of) 
the image, and it can be stored in a cache memory. Consequently, when plural pictures are to be 
displayed simultaneously, it is possible to allot more bandwidth to the entity with URL data 
taken as a control. 
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In step 85, the interframe data are removed and are provided to the computer structural 
elements in the receiving platform (see Figure 1). In step 87, according to the present 
embodiment of the present invention, the data are provided to the CPU that executes a dynamic 
URL control routine according to the embodiment of the present invention. Normal TV pictures 
are provided from the frame data in step 89 by means of conventional TV structural elements of 
the reception conversation system. 

In step 91, data from the interframe region are processed for highlighting the identified 
entity. Here, highlighting is performed in step 93. In step 95, the viewer provides pointer input to 
activate the cursor on the TV screen. The input is processed in step 97. In step 99, the viewer 
moves the cursor to the area of the highlighted entity image. In step 101, the viewer starts the 
selection input, and this is processed in step 103. In step 105, the highlighted entity is selected. 
This selection starts web access by the computer structural elements in the receiving/displaying 
system. 

In step 107, the network interface module (NIM) is initialized, dial-up is performed, and 
internet access is provided to the receiving system. The NIM may be an analog or digital modem, 
cable modem, satellite modem, either of computer network cards (lObT) and (lOObT), a token 
ring or the like, as well as many other techniques for accessing the internet. 

In step 109, the dynamic URL correlated with the highlighted entity is provided to the 
internet, and the related web page is downloaded. In step 1 1 1, the web page is displayed itself or 
is displayed on the window being displayed. In step 113, actions that may be performed by the 
viewer are displayed on the web page, and by means of any method known to specialists, the 
related information is selected, and jumping to the related site on the WWW occurs, followed by 
conversation with the web page. In step 1 15, the viewer turns OFF the web display by selecting 
the well known close box or with any available scheme known to the viewer. 

As explained above, for an advertisement, delayed repeated program, or the like, display 
is mainly correlated with the prerecorded contents with respect to the TV display. For such feed 
materials, in order to correlate one or several images in the adjacent frames to an URL, editing is 
performed by adding the necessary number of bytes of information between frames to each 
frame. The method for performing such editing is well known to specialists of TV and data 
transmission technology. The same technology may also be used with VCR media, CD-ROM 
media at the same time. Known methods for recording and transmitting still or dynamic images 
for succeeding image displays are not excluded. For example, the MPEG data format contains a 
means for including such data. 

Figure 3B is a flow chart illustrating the general steps in preparing a TV broadcast 
program with a dynamic URL according to an embodiment of the present invention. In step 73, 
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an entity, such as said BMW emblem in the aforementioned example, is identified in the scene as 
to whether it should be broadcast or correlated with a dynamic URL. 

In step 75, the identified entity is correlated with an URL. Here, the URL is one with 
respect to a web page maintained for providing additional information correlated with the 
identified entity or a web page correlated with a TV broadcast using a conventional method. 

In step 77, the position and the area range of the identified entity are often identified with 
respect to the broadcast frame pertaining to the geometric shape of the frame. That is, the 
position and range of the entity on the display screen are identified. 

In step 79, the URL that correlates the data that define the position and area range of the 
identified entity to the entity are recorded in the data region from the image data with respect to 
the display frame in the data stream for broadcast. The separated data region is correlated with 
the frame. 

In step 81, steps 1-4 are performed repeatedly with reference to all of the frames for 
which the entity should be correlated with the dynamic URL. In many cases, since one entity is 
repeated in a relatively large number of frames, the viewer has sufficient time to react to the 
dynamic entity for selection. For a single program or broadcast, several entities may be 
correlated with one or several URLs. 

Any entity may be a candidate, and there are many possibilities. For example, persons 
may be selected, objects or man-made products may be selected, or an additional icon or image 
may be selected. In certain cases, the entity to be correlated with the identified URL is actually 
added to the scene by additional data in the same region as that used with respect to the URL. In 
certain embodiments, just as aforementioned, a single (point) position is identified with respect 
to the entity, a circular area, square area or another general area is correlated with the entity, and 
the general region should be the area selected by the viewer to start the dynamic URL. 

The present invention is not limited to prerecorded edited image data. The effects of the 
present invention may also be displayed using the unique method for sports events or similar live 
broadcasts. In a live broadcast, the interframe data, including the URL correlated with the image 
should be essentially inserted in real time or inserted with a minimum delay. This means that for 
the camera and transmitter as the processor of the correlated images and data, a method should 
be present to determine which depicted object or person is the object or person related to the 
URL, and the device must produce data that should be inserted in each frame. 

In order to realize a real-time live broadcast with a dynamic URL, it is sufficient that the 
relationship and position of the object or person to be correlated with the URL is notified to an 
independent image device. In certain embodiments, this device takes the form of an IR 
transmitter or wireless beacon transmitter or similar broadcast device. 
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The explained broadcast device can attain two substantial objectives in transmission to 
the image device. The first thing is the position, and the position and area range data in the 
interframe region corresponding to said information is inserted in the image device, and the 
second thing is the URL itself. Since the URL is shared for events other than objects and 
persons, the URL may be automatically inserted in this embodiment. In other embodiments, a 
prescribed URL is added with respect to a tracked object or person. In the case of a real-time live 
broadcast, the aforementioned technology in which the point position is identified and the 
standard area range is inserted provides a convenient method. 

There are many convenient methods by which a TV camera or another device can 
provide the necessary information, including the dynamic URL, for the depicted persons and 
objects. Said IR and wireless beacon are merely two examples among many types. In one 
embodiment, a bar code may be applied to clothes of players, and said bar code can be read by 
the image device. As another embodiment, one may also use X-rays, a UHF broadcast or other 
substitute broadcast technologies. Any method for transmitting information to the remote side 
may be adopted. 

Specialists may adopt various substitute embodiments as embodiments of the present 
invention as long as the gist and range of the present invention are observed. For example, with 
reference to Figure 1, various types of devices may be adopted appropriately in embodiment of 
the present invention. For embodiment of the present invention, in all cases, not all of the 
structural elements shown in Figure 1 are necessary, and only certain selected structural elements 
in the embodiment are needed. For example, it is not necessary to have a CD-ROM drive and 
hard disk drive in all embodiments, yet these drives are used in some embodiments. 

In addition to the aforementioned, the TV broadcast from any one remote site may be a 
live broadcast or a broadcast of prerecorded programs, as explained in the aforementioned 
embodiments. However, the present invention may also be adopted using the following scheme: 
other methods for feeding a TV signal to a display, methods with a unifying capability as a 
response to a dynamic URL. For example, by means of a signal stored in a hard disk drive, or by 
means of a signal provided to a CD-ROM disk for reproduction by the CD-ROM drive shown in 
Figure 1, a TV program having a correlated URL is provided to a videotape used in a VCR. The 
present invention is very broad since it includes any means for providing a signal for TV display, 
and, by means of a signal unified with a dynamic URL, the user browses the internet WWW, 
and, as a result, conversation is possible with a highlighted entity in a frame of the TV display. 

In addition, in order to remove the interframe data and to execute browsing as a response 
to a dynamic URL, many methods can provide the functionality of the necessary control routine 
for said device. It is well known in computer technology that a prescribed functionality can be 
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provided by a control routine written by various methods. As a unique feature, instead of a 
prescribed order of a code, functionality exists in providing code to the device. 

In the above, several embodiments were explained. However, the present invention can 
be adopted in many potential applications. Application of the present invention in such case is 
taken as a novel embodiment of the present invention instead of another new invention. 

In addition, there are many formats for display of a web page while a TV program is 
simultaneously displayed. There are many modified examples that can simply verify the range of 
the present invention. 
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Figure 3A 



Key: a Interframe data 

b Frame data 

83 Reception of a data stream that transmits image frame data and a dynamic URL in 

separate data regions 

85 Interframe data excluding the interframe data for the structural elements of 

computer 

87 Execution of dynamic URL code routine 

89 Display of image frame data providing television display 

91 Processing of data for highlighting identified entity 

93 Holding of entity with TV image identified and highlighted 

95 User pointer input 

97 Display and move cursor 

99 Viewer moves cursor to a highlighted entity image region 
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1 0 1 Selection of input by user 

1 03 Start of internet access by means of highlighted URL 

1 05 Selection of highlighted entity by the user 

1 07 Initialization of modem and completion of dial-up 

1 09 Download of web page 

1 1 1 Display of web page 

1 1 3 Action taken by viewer via web page 

115 Cancel browsing 
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Figure 3B 



Key: 73 Identification of entity in page to be broadcast 

75 Correlation of identified entity to universal resource locator 

77 Identification of position and area range of identified entity in frame with respect 

to geometric shape of the frame (position and range on display screen) 
79 Setting of the information from step 3 in the broadcast data stream in the region 

separate from the image data with respect to the frame 
8 1 Repeat 1 -4 with respect to all of the frames with the dynamic URL in the data 

stream 
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